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Abstract 

Humans  confront  the  unexpected  every  day,  deal  with  it,  and  often  learn  from  it. 
AI  agents,  on  the  other  hand,  are  typically  brittle — they  tend  to  break  down  as 
soon  as  something  happens  for  which  their  creators  did  not  explicitly  anticipate. 
The  central  focus  of  our  research  project  is  this  problem  of  brittleness  which  may 
also  be  the  single  most  important  problem  facing  AI  research.  Our  approach  to 
brittleness  is  to  model  a  common  method  that  humans  use  to  deal  with  the 
unexpected,  namely  to  note  occurrences  of  the  unexpected  (i.e.,  anomalies),  to 
assess  any  problem  signaled  by  the  anomaly,  and  then  to  guide  a  response  or 
solution  that  resolves  it.  The  result  is  the  Note-Assess-Guide  procedure  of  what 
we  call  the  Metacognitive  Loop  or  MCL.  To  do  this,  we  have  implemented  MCL- 
based  systems  that  enable  agents  to  help  themselves;  they  must  establish 
expectations  and  monitor  them,  note  failed  expectations,  assess  their  causes,  and 
then  choose  appropriate  responses.  Activities  for  this  project  have  developed  and 
refined  a  human-dialog  agent  and  a  robot  navigation  system  to  test  the  generality 
of  this  approach. 


Activities 


The  research  has  focused  on  testing  the  hypothesis  that  a  properly  designed  metacognitive  loop 
can  be  both  domain-  and  anomaly-general,  by  implementing  it  in  a  (partly  physical,  partly 
simulated)  world  containing  many  of  the  most  challenging  and  important  facets  of  the  general 
infosphere  problem,  which  involves  multiple  autonomous  agents,  which  can  gather  and  transmit 
data  as  well  as  request,  receive  and  process  data.  We  have,  in  fact,  implemented  the 
Metacognitive  Loop  in  several  distinct  types  of  systems:  a  reinforcement  learner,  a  human- 
computer  dialogue  agent,  a  tank  game,  a  robot  navigation  system,  and  a  commonsense 
(nonmonotonic)  reasoner.  In  each  case,  the  performance  of  the  system  was  enhanced  by  MCL 
mechanisms.  Activities  for  the  current  project  have  developed  and  refined  a  human-dialog  agent 
and  a  robot  navigation  system.  The  former  is  called  ALFRED  and  the  latter  the  simulated  Mars 
rover. 

ALFRED  is  a  universal  interfacing  agent  that  accepts  a  set  of  English  sentences  and  translates 
them  into  commands  appropriate  for  different  domains.  The  original  ALFRED  had  no  parsing 
capacity  (parsing  was  conducted  by  an  outside,  off-the-shelf  system),  and  its  knowledge  of 
language  in  general  was  extremely  limited.  In  this  project  we  developed  ALFRED  2.0,  the  next 
generation  of  this  interfacing  agent.  ALFRED  2.0,  like  its  predecessor,  is  written  in  Alma,  a 
Prolog-like,  declarative  language  (Alma  itself  is  written  in  Prolog).  What  makes  Alma  unique  is 
its  step-wise  temporal  reasoning  capability.  Alma  keeps  track  of  time  by  counting  off  steps  (or 
time  units).  Each  step  in  a  logical  derivation  corresponds  to  an  Alma  step. 

For  instance,  at  step  1,  Alma  can  accept  that  P,  P  ->  Q,  and  Q  ->  R  is  true.  Then  each  step  in  the 
derivation  of  R  takes  a  time  step.  In  a  language  like  Prolog,  R  would  be  concluded  immediately 
instead.  The  step-by-step  reasoning  was  designed  to  accommodate  contradictions  in  an  agent’s 
knowledge  base.  Thus,  ALFRED  2.0  is  a  semantic  ontology,  i.e.  a  characterization  of 
knowledge,  written  in  the  Alma  formalism.  Alma  is  the  reasoning  engine  that,  step-by-step, 
expands/changes  ALFRED’S  knowledge  base.  This  knowledge  consists  of  linguistic  information 
(mostly)  but  was  designed  so  that  general  knowledge  could  be  added  easily.  ALFRED  2.0  was 
created  to  provide  a  natural  language,  bidirectional  interface  to  an  open  set  of  domains.  It  acts  as 
a  translator,  taking  typed  English  utterances  as  input  (although  it  can  also  take  voice  input  from  a 
speech-to-text  software  package  called  Dragon),  and  outputs  the  domain  appropriate  command 
string. 

We  have  also  developed  a  substrate  for  simulating  discrete  worlds  in  which  metacognitive  agents 
can  act.  Upon  this  substrate,  we  have  implemented  a  basic  Mars  domain  in  which  simulated 
Mars  Rovers  can  be  deployed  to  conduct  basic  scientific  missions.  The  simulated  Rovers  possess 
planners  for  bottom-up  navigation  and  for  high-level  mission  activities,  and  they  have  basic 
monitor  and  control  capabilities  that  are  integrated  with  MCL.  Coupled  with  control  over 
perturbations  to  the  domain,  the  Mars  Rover  provides  a  simple,  flexible,  and  extensible  platform 
for  developing  the  monitor  and  control  portions  of  the  MCL  ontologies  and  a  test  bed  for 
demonstrating  the  general  efficacy  of  MCL.  Further,  the  Mars  domain  provides  basic  TCP/IP 
command-dispatching  agents  and  goal  monitoring  agents  to  provide  a  common  interface  for 
introducing  goals  and  perturbations  to  the  simulated  environment  and  scoring  the  progress  of 
Mars  Rovers,  respectively.  We  also  have  configured  the  same  MCL  software  to  connect  to 
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physical  mobile  robots  (at  the  Naval  Research  Lab,  at  Franklin  &  Marshall  College,  and  at  the 
University  of  Maryland,  Baltimore  County). 

Finally,  we  have  an  overarching  hypothesis:  that  the  identical  core  MCL  code  will  serve  to  guide 
multiple  applications,  and  in  others  still  to  be  designed.  As  such  MCL  is  a  monitoring -and- 
control  module  that  can  be  attached  to  a  given  “host”  system,  thereby  yielding  an  improved 
system.  The  team  has  designed  and  built  a  prototype  version  of  generalized  MCL  and  attached  it 
to  various  host  systems  including  search-and-rescue  robots  and  Mars-style  rovers  (both  real  and 
simulated),  a  simulated  air  traffic  control  system,  and  a  natural-language-based  human-computer 
interaction  system. 

Testing  our  overarching  hypothesis  requires  producing  ontologies  of  indications,  failures,  and 
repairs  for  these  domains  and  extracting  common  core  structure.  We  did  this  for  human- 
computer  dialog  systems  with  the  goal  of  allowing  the  computer  to  automatically  determine 
when  the  human  is  dissatisfied  with  the  dialog  and  to  effect  a  repair.  The  ontologies  were 
constructed  by  reviewing  the  literatures  on  human  and  human-computer  dialog,  and  by  running 
experiments  with  human  subjects  interacting  with  AIML  chatbots.  The  output  of  the  process  was 
a  complete  set  of  ontologies,  with  linkages  both  within  and  between  ontologies  suitable  for 
ingest  by  the  MCL  implementation. 

We  also  developed  simulated  search  and  rescue  robots  that  can  avoid  obstacles,  locate  items  of 
interest  and  gather  these  items.  An  increase  in  number  of  collisions  (encounters  with  danger) 
causes  an  expectation  violation  that  a  metacognitive  component  notes.  This  expectation  violation 
causes  the  metacognitive  component  to  adjust  different  parameters  like  the  safe  distance  from 
obstacle,  and  the  speed  and  time  spend  on  path  planning.  The  ontology  of  indications,  failures 
and  responses  that  the  on-board,  metacognitive  component  of  the  robots  use  is  compatible  with 
MCL. 

We  have  built  specialized  controllers  for  two  Corobots,  integrated  the  controllers  with  the  central 
communication  system  ("RonCon"),  and  designed  and  deployed  appropriate  test  courses  for  the 
robots.  We  also  designed  and  implemented  a  new  algorithm  for  color  classification  that  shows 
significant  robustness  across  lighting  conditions.  This  is  currently  being  prepared  for  publication. 

At  UMBC  we  developed  the  core  ontology-based  MCL  implementation,  the  monitor  and  control 
(“MonCon”)  infrastructure  that  connects  distributed  host  systems  and  humans  to  explore  the 
behavior  of  MCL  in  multi-agent  environments,  and  the  analogous  infrastructure  (RonCon) 
suitable  for  robotic  agents.  We  also  implemented  the  Mars  Rover  domain  and  integrated  it  with 
MonCon.  Finally,  the  work  with  AIML  chatbots,  including  human  subjects  experiments,  was 
done  at  UMBC,  as  was  most  of  the  development  of  the  dialog  ontologies  for  MCL. 

Underlying  the  MCL  approach  is  a  time- sensitive,  contradiction-tolerant  logical  reasoning 
engine  called  the  active  logic  machine  (ALMA).  We  have  formalized  a  semantics  for  a  general 
version  of  the  underlying  logical  formalism,  active  logic.  Central  to  active  logic  are  special  rules 
controlling  the  inheritance  of  beliefs  in  general  (and  of  beliefs  about  the  current  time  in 
particular),  very  tight  controls  on  what  can  be  derived  from  direct  contradictions  (P&-P),  and 
mechanisms  allowing  an  agent  to  represent  and  reason  about  its  own  beliefs  and  past  reasoning. 
Furthermore,  inspired  by  the  notion  that  until  an  agent  notices  that  a  set  of  beliefs  is 
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contradictory,  that  set  seems  consistent  (and  the  agent  therefore  reasons  with  it  as  if  it  were 
consistent),  we  introduce  an  “apperception  function”  that  represents  an  agent’s  limited  awareness 
of  its  own  beliefs,  and  serves  to  modify  inconsistent  belief  sets  so  as  to  yield  consistent  sets. 
Using  these  ideas,  we  introduced  a  new  definition  of  logical  consequence  in  the  context  of  active 
logic,  as  well  as  a  new  definition  of  soundness  such  that,  when  reasoning  with  consistent 
premises,  all  classically  sound  rules  remain  sound  in  our  new  sense.  However,  not  everything 
that  is  classically  sound  remains  sound  in  our  sense,  for  by  classical  definitions,  all  rules  with 
contradictory  premises  are  vacuously  sound,  whereas  in  active  logic  not  everything  follows  from 
a  contradiction. 

The  process  of  rationally  revising  beliefs  in  the  light  of  new  information  is  a  topic  of  great 
importance  and  long-standing  interest  in  artificial  intelligence.  Moreover,  significant  progress 
has  been  made  in  understanding  the  philosophical,  logical,  and  computational  foundations  of 
belief  revision.  However,  very  little  research  has  been  reported  with  respect  to  the  revision  of 
other  mental  states,  most  notably  propositional  attitudes  such  as  desires  and  intentions.  In  this 
project,  we  presented  a  first  attempt  to  formulate  a  general  framework  for  understanding  the 
revision  of  mental  states.  We  developed  an  abstract  belief-desire-intention  model  of  agents,  and 
introduce  a  notion  of  rationality  for  this  model.  We  then  presented  a  series  of  formal  postulates 
characterizing  the  processes  of  adding  beliefs,  desires,  and  intentions,  updating  costs  and  values, 
and  removing  beliefs,  desires,  and  intentions.  We  also  investigated  the  computational  complexity 
of  several  problems  involving  this  abstract  model. 

We  have  also  made  advances  in  MCL-related  natural  language  understanding.  Language  not 
only  refers  to  objects  and  events  in  the  world,  but  it  also  can  refer  to  language  constituents 
themselves.  This  metalinguistic  phenomena  is  captured  in  the  use-mention  distinction.  In 
particular,  we  are  able  to  recognize  mentioned  language:  that  is,  tokens  (e.g.,  words,  phrases, 
sentences,  letters,  symbols,  sounds)  produced  to  draw  attention  to  linguistic  properties  that  they 
possess.  Evidence  suggests  that  humans  frequently  employ  the  use-mention  distinction,  and  we 
would  be  severely  handicapped  without  it;  mentioned  language  frequently  occurs  for  the 
introduction  of  new  words,  attribution  of  statements,  explanation  of  meaning,  and  assignment  of 
names.  Moreover,  just  as  we  benefit  from  mutual  recognition  of  the  use-mention  distinction,  the 
potential  exists  for  us  to  benefit  from  language  technologies  that  recognize  it  as  well.  With  a 
better  understanding  of  the  use-mention  distinction,  applications  can  be  built  to  extract  valuable 
information  from  mentioned  language,  leading  to  better  language  learning  materials,  precise 
dictionary  building  tools,  and  highly  adaptive  computer  dialogue  systems. 

Three  specific  contributions  were  made.  The  first  is  a  framework  for  identifying  and  analyzing 
instances  of  mentioned  language,  in  an  effort  to  reconcile  elements  of  previous  theoretical  work 
for  practical  use.  Definitions  for  mentioned  language,  metalanguage,  and  quotation  have  been 
formulated,  and  a  procedural  rubric  has  been  constructed  for  labeling  instances  of  mentioned 
language.  The  second  is  a  sequence  of  three  labeled  corpora  of  mentioned  language,  containing 
delineated  instances  of  the  phenomenon.  The  corpora  illustrate  the  variety  of  mentioned 
language,  and  they  enable  analysis  of  how  the  phenomenon  relates  to  sentence  structure.  Using 
these  corpora,  inter-annotator  agreement  studies  have  quantified  the  concurrence  of  human 
readers  in  labeling  the  phenomenon.  The  third  contribution  is  a  method  for  identifying  common 
forms  of  mentioned  language  in  text,  using  patterns  in  metalanguage  and  sentence  structure. 
Although  the  full  breadth  of  the  phenomenon  is  likely  to  elude  computational  tools  for  the 
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foreseeable  future,  some  specific,  common  rules  for  detecting  and  delineating  mentioned 
language  have  been  shown  to  perform  well. 

Given  these  results,  we  are  beginning  two  new  research  thrusts  to  extend  the  current  research 
reported  here.  First  we  have  started  to  examine  the  psychological  and  cognitive  neuroscience 
correlates  to  human  metacognitive  processes  similar  to  the  MCL  mechanism.  Initial  studies  by 
Anderson  at  Franklin  &  Marshall  have  examined  neural  reuse  and  functional  connectivity  in  the 
brain  and  how  all  of  cognition,  even  metacognition,  is  grounded  in  concrete  experience.  Second 
we  have  begun  a  long-term  project  to  apply  our  results  to  the  design  of  a  large  cognitive 
architecture  that  focuses  upon  the  interaction  of  cognition  and  metacognition.  The  system  is 
called  MIDCA  for  the  Metacognitive  Integrated  Dual-Cycle  Architecture.  These  two  new 
projects  show  much  early  promise  and  would  not  have  come  about  without  the  foundational 
research  this  AFOSR  funding  provided. 

Contributions 

This  work  modifies  and  enhances  decades-long  efforts  to  design  logic-based  commonsense¬ 
reasoning  intelligent  systems.  Chiefly,  we  incorporated  real-time,  natural-language,  and 
inconsistency-tolerant  aspects  into  the  system  capabilities,  by  means  of  a  re-conceptualization  of 
common  sense.  Namely,  we  investigated  autonomous  error-correction  (note,  assess,  and  respond 
to  anomalies)  as  a  key  element  heretofore  missing  in  this  area.  Systems  so-enhanced  appear  to  be 
far  more  robust  and  adaptive  in  the  face  of  unanticipated  circumstances. 

Training  and  Development 

Various  students  and  postdocs  were  fully  enmeshed  in  this  work.  Both  the  development  and  the 
testing  of  the  MCL  system  has  allowed  us  to  engage  students  at  many  different  levels  of  the 
research,  from  theoretical  work,  to  programming,  to  physically  setting  up  robotic  test  courses.  In 
addition,  many  students  participated  in  high-level  conceptual  work  as  the  project  evolved  to  later 
stages.  Three  PhD  students  finished  their  research  with  support  from  this  grant  and  have 
successfully  graduated.  Two  postdoctoral  fellows  received  valuable  guidance  under  this  grant 
and  contributed  significantly  to  the  research  program.  The  high-school  students  involved  in  the 
project  finished  their  parts  of  the  project  successfully,  in  the  process  learning  a  great  deal  about 
AI,  some  of  it  quite  different  from  what  they  had  imagined. 

Outreach 

During  early  2011,  we  established  a  new  seminar  series  at  the  University  of  Maryland  College 
Park  campus.  The  Maryland  Metacognition  Seminar  (www.cs.umd.edu/active/MetaCogSeminar) 
reaches  out  to  institutions  in  the  greater  DC  metropolitan  area  to  attract  faculty  and  students  that 
engage  or  are  interested  in  research  related  to  important  issues  in  human  and  machine 
metacognition  and  metareasoning.  Don  Perlis  gave  the  inaugural  presentation  on  March  4,  2011, 
and  thirteen  additional  research  presentations  followed  up  to  the  current  time.  The  ongoing  series 
provides  an  interdisciplinary  forum  for  both  preliminary  and  mature  research  results  and 
normally  occurs  on  a  monthly  basis. 

The  results  of  our  research  and  all  of  the  publications  that  have  originated  with  this  grant  are 
available  on  the  web  for  public  release.  See  www.cs.umd.edu/proiects/active. 
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